Scaling Policy Preferences from Coded Political Texts

Scholars wanting to estimate substantive quantities of interest, for example policy positions, from political texts typically apply a coding scheme to discrete text units such as words or sentences. Scales of policy positions, for example a left-right scale of economic policy, are typically built from the relative frequencies of text units coded into different categories. In this paper we reexamine the theoretical and linguistic basis for such scales, proposing a new alternative. This is based the logarithm of odds-ratios and is consistent with underlying political and linguistic mechanisms typically assumed to underlie text generation. We contrast this scale with previous approaches using coded text data from the Comparative Manifesto Project (CMP). We show that the logit scale we propose avoids widely acknowledged flaws in previous approaches. We validate the new scale using independent expert surveys of policy positions. Using existing CMP data, we show how to estimate more different policy dimensions, for more years, than has been possible before. We make this new dataset available, along with estimates of uncertainty for each measure. Finally, we draw some conclusions about the future design of coding schemes for political texts.